[SVS] Refactor SVS element size calculations and block size handling#980
Open
rfsaliev wants to merge 5 commits into
Open
[SVS] Refactor SVS element size calculations and block size handling#980rfsaliev wants to merge 5 commits into
rfsaliev wants to merge 5 commits into
Conversation
🛡️ Jit Security Scan Results
✅ No security findings were detected in this PR
Security scan by Jit
|
ethanglaser
reviewed
Jun 17, 2026
|
|
||
| cmake_dependent_option(SVS_SHARED_LIB "Use SVS pre-compiled shared library" ON "USE_SVS AND GLIBC_FOUND AND SVS_LVQ_SUPPORTED" OFF) | ||
| if (CMAKE_CXX_COMPILER_ID MATCHES "Clang") | ||
| if(GLIBC_VERSION VERSION_GREATER_EQUAL "2.31" AND CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL "21.0") |
Collaborator
There was a problem hiding this comment.
Suggested change
| if(GLIBCXX_VERSION VERSION_GREATER_EQUAL "11") | |
| set(SVS_URL "https://github.com/intel/ScalableVectorSearch/releases/download/nightly/svs-shared-library-reduced-clang21-gcc11-2026-06-17-1412.tar.gz" CACHE STRING "SVS URL") |
Codecov Report✅ All modified and coverable lines are covered by tests. Please upload reports for the commit fcc801d to get more accurate results. Additional details and impacted files@@ Coverage Diff @@
## main #980 +/- ##
==========================================
- Coverage 97.12% 97.10% -0.03%
==========================================
Files 141 141
Lines 8179 8179
==========================================
- Hits 7944 7942 -2
- Misses 235 237 +2 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
alonre24
previously approved these changes
Jul 1, 2026
alonre24
left a comment
alonre24
left a comment
Collaborator
There was a problem hiding this comment.
LGTM - did we confirm that there is no regression in terms of memory consumption in benchmarks?
alonre24
reviewed
Jul 1, 2026
| if(GLIBC_VERSION VERSION_GREATER_EQUAL "2.28") | ||
| if(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL "14.0") | ||
| set(SVS_URL "https://github.com/intel/ScalableVectorSearch/releases/download/v0.3.0/svs-shared-library-0.3.0-reduced-gcc14.tar.gz" CACHE STRING "SVS URL") | ||
| set(SVS_URL "https://github.com/intel/ScalableVectorSearch/releases/download/nightly/svs-shared-library-reduced-gcc14-2026-06-18-1414.tar.gz" CACHE STRING "SVS URL") |
Collaborator
There was a problem hiding this comment.
Do we want to merge it as it is (using the nightly version) or bump to an official svs release?
rfsaliev
force-pushed
the
rfsaliev/svs-blocksize-elements
branch
from
465995d to
fcc801d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Describe the changes in the pull request
This pull request refactors and improves the way element sizes and block sizes are calculated and used in the SVS storage traits, enhancing clarity and modularity in the code. The changes introduce clearer separation between primary, residual, and secondary element sizes, update the block size calculation to use a new structure, and simplify related test logic.
Which issues this PR fixes
This PR fixes index size estimation issues for SVSIndex by using new
blocksize_elementsparameter introduced in intel/ScalableVectorSearch#344.TODO:
Main objects this PR modified
Mark if applicable
Note
Medium Risk
Changes SVS blocked allocation and memory estimation for quantized indexes and bumps the fetched shared library; incorrect blocking could affect memory use or stability, but scope is limited to SVS build/runtime paths.
Overview
Aligns SVS integration with ScalableVectorSearch v0.3.1 (prebuilt tarball URLs in
cmake/svs.cmake, including a simpler Clang path that only selects the gcc11 binary when libstdc++ ≥ 11).LVQ / LeanVec storage now splits byte layout into
primary_element_size,residual_element_size, andsecondary_element_size, and uses the max of the relevant parts when sizing blocks.SVSBlockSizereturnssvs::data::BlockingParameters(power-of-two byte block plusblocksize_elementsfrom the VecSim block count), matching upstream blocking behavior and fixing index size estimation.Tests drop LVQ-specific dimension workarounds in
testSizeEstimation(tighter 1% tolerance) and set an explicitblockSizein tieredrunGCAPIso GC exercises at least two vector blocks.Reviewed by Cursor Bugbot for commit fcc801d. Bugbot is set up for automated code reviews on this repo. Configure here.